Resolution of an Orbital Issue: 
A Designed Experiment 


Lisa Huccdleston 
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Background 


* During STS-1 14, on , an issue came up that required the astronauts to 
take a space walk visit to the underneath side of the orbiter 

* This is story of the “Gap Filler” and how we used an advanced 

statistical method to prevent it from happening again.... 

* So let’s see what happened. 


http://video.qooqle.com/videoplav?docid=1 1 83929501 51 1 634226# 
OR 

http://www.voutube.com/watch?v=KmQp7Ab3Bds 


On-orbit issue 


What was the issue? 

Who cares if a hunk of ceramic sticks: up? 
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There was extreme interest in not allowing this to happen again... so 
we used a designed experiment to determine the best way to address 
it 
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Background For Design of Experiments (DOE) 


■ DOE is a systematic approach to investigation of a system or process. A 
series of structured tests are designed iir which planned changes are made to 
the input variables of a process or system. The effects of these changes on a 
pre-defined output are then assessed. 

■ DOE is a formal method of maximizing information gained while minimizing 
resources required. 
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Background For Design of Experiments (DOE) 


DOE can be used to find answers in situations such as: 

■ What is the main contributing factor to a problem? 

■ How well does the system/process perform in the presence of noise? 

(defined as randomness in response due to uncontrolled variation) 

■ What is the best configuration of factor values to minimize variation in a 
response or outcome? 


Information extracted from “Design of Experiments in brief’, http://www.thequalityportal.eom/q know02.htm 


Background For Design of Experiments (DOE) 

How to perform it 

- Identify the input variables and the response (output) that is to be 
measured. 

- For each input variable, a number of levels are defined that represent 
the range for which the effect of that variable is desired to be known. 

- Produce an experimental plan which tells the experimenter where to set 
each test parameter for each run of the test. 

- Measure the response for each run. 

- Look for differences between response (output) readings for different 
groups of the input changes. 

- Attribute the differences to the input variables acting alone (called a 
single effect) or in combination with another input variable (called an 
interaction). 


Information extracted from “Design of Experiments in brief, http://www.thequalityportal.eom/q know02.htm 


Design of Experiments (DOE) Example 

Gap Filler Pre Process Change Testing 

(These samples were subjected to the V 2 lb. pull load which was the 
spec requirement prior to the process change. The current pull 
load requirement is 5 lbs.) 

• Pre-Process Change Testing 

- Bonded 20 1-ply per P-602 spec 

- Bonded 20 2-ply per P-602 spec 

- Bonded 20 4-ply per P-602 spec 

• Results 

- 1 ply failure rate: 12/20 failed = <60% 

- 2 ply failure rate: 9/20 failed = 45% 

- 4 ply failure rate: 2/20 failed = 10% 


• This showed us that the increased irumber of plies appeared to 
increase bond reliability. 
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Design of Experiments (DOE) Example 


Observations from Pre-test which also may have affected 
the bondline integrity: 

• lack of RTV on filler bar 

- Armalon RTV squeeze out 

• poor pressure application 

• sidewall bonds 

• gap filler protrusion after bond 

• non-uniform load during pull test 

• partial debonds 

• multi-ply gap fillers may not deflect when debonds are present 
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Sandwich 0.05” Dip 



Design of Experiments (DC(E) Example 


• This is the kind of thing we all learnecc about in engineering school.... 

• So.... the science is well understood 

• The methods are well understood 

• The selection techniques are well umcerstood 

• the primary difficulty in performing a set up and test like this is getting 

management to understand what you’re doing; why you’re doing it that way; 
and why it’s important to follow the rules. 
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Design of Experiments ((DOE) Example 

Qualification Test of Process Enhancements 


-144 gap fillers distributed as follows 


Factors 

Levels 

Technicians 

6 levels - randomly selected from 
population of technicians with gap filler 
cert 

Inspectors 

2 levels- randomly selected from 
population of TPS quality inspectors 

Fill type 

2 levels- complete or nominal 

Lengths 

4 levels -1"”, 3”, 5”, and 7” 

* Plys/flat panel 

3 levels - 1, 3, and 4 ply 

* Plys/curved panel 

3 levels - 2, 5, and 6 ply 
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Design of Experiments (DOE) Example 
Qualification Test Results 

• Ordinary Least Squares Regression was performed on the 
142 data points to predict and hypothesize on the effects that 
the gap filler process improvements may have brought to gap 
filler bonding at KSC 


Design of Experiments (DOE) Example 



Design of Experiments (DOE) Example 


Source 

Nparm 

DF 

Sum of Squares 

F Ratio 

Prob > F 

Rank 

Technician 

5 

5 

1152.939 

1.9425 

0.0917 

3 

QC 

1 

1 

107.005 

0.9014 

0.3442 

4 

Length (in.) 

1 

1 

54580.025 

459.7903 

^.OOOl " 

1 

Ply 

5 

4 

9198.389 

19.3722 

<.0001 


Panel 
(0 = flat, 

1 = curved) 

1 

0 

0.000 

NA 

^ 

NA 

Complete or 
Nominal? 

(0 = nominal, 
1 = complete) 

1 

1 

1.983 

0.0167 

0.8974 
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The smaller 
the number 
the less likely 
the observed 
effect 

occurred by 
chance. 



Design of Experiments (DOE) Example 

What does this mean? 

• Ply, and length have the greatest effect on the pull to failure load. 

• Fill type (complete or nominal), technician, or QC do not have as great an 
effect. 

• The effect of the panel could not be statistically determined since ply and 
panel type were not independent. H-owever, examination of the following 
figures indicates that IML curvature does have an effect on bond strength. 

• The following is the overall summary of the 142 data points: 


Average Pull Value (lbs.): 

41.09 

Minimum value (lbs.): 

3 

Maximum value (lbs.): 

118 

% Complete Fill: 

35.42% 

% Nominal Fill: 

64.58% 

% Curved: 

43.75% 

%Flat: 

56.25% 

% Failure Rate: 

0.70% 
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Design of Experiments (DOE) Example 
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Ply within Length (in.) 
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Design of Experiments (DOE) Example 



i i i i i i i i 

1.5 3 5 7 

Panel (0 = flat, 1 = curved) within Length (in. 
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Design of Experiments (DOE) Example 


Major Process Enhancements 

- Gap Filler length limited to 1 .5 to 7 inches 

- Clarified the number of pull loops required for varying lengths 

- Moved location of pull loop holes above OML trim line and increase hole 
diameter 

- Added Sandwich method for gap filler installation based on OPF technician 
recommendations and process testing 

- Added description for bonding pressure using tape method on the high gap 
filler 

- Added requirement to sever any siccewall bonds using 0.006 inch thick metal 
shim 

- Increased the pull loads to 5.0 +1.0/-0.0 pounds 

- Trim gap filler flush to 0.12 inch recessed after pull test 
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Design of Experiments (DOE) Example 
Conclusion of gap filler experiment 

• New process allows for an average gap filler pull to failure 
value of 41 lbs. 

- Range of pull to failure values range from 3 lbs. To 1 18 lbs. 

• Length and to a lesser degree, ply, have the greatest effect 
on the pull to failure load 

- Curved or flat panel effect cannot be determined statistically but from 
observation of trend on previous charts, IML curvature does seem to 
have an effect on bond strength. 
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Conclusion 


* So what did we get out of all of tlhis? 


- Proven enhancements to the process 

- Better understanding of the installation process 


- AND — 

- Not a single gap filler issue during remaining missions for Space 
Shuttle 

- AND 

- Better management support for use of advanced statistical 
methods in the operational environment. 
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